Hung up momentarily on the discourse re: "human writing/code/art is on average pretty bad, so the median AI output is better than the median human work." Writing on phone to get the thought out without proofreading so sorta rough.
Say the average human craft is something like a pair of worn denim jeans. Its not perfect, it doesn't do everything you want, but its form is honest. You run your hand along the fabric and it holds its shape, continues smoothly where you expect it to. Your finger finds a rivet, the stitching at the seams, a small hole from wear. The hole is only a hole where it is a hole - you know where it ends, so you know to take care and handle it gently around the area. You start to worry and rip it, the rip extends along the grain of the fabric. It degrades in a predictable path. The fabric is a meshwork of thread, an affordance that supports stitching, mending - if I bring the two edges of the tear together and sew them, then they will rejoin, with a seam, with an amended shape, but still rejoin. In their normal use, the pants are pants, and the edges of pathological pants behavior are visible but predictable.
The average AI craft might be some kind of alien LED-laden fast fashion garment. Cheaper, higher tech, maybe even more adventurous and exciting. They aren't pants per se, its not exactly clear how its intended to be worn. You can put your legs through some of the holes and it covers most of your body, but instead of a fly there's a long conical sleeve you have to knot closed. There are collars around the knees. Hundreds of belt loops ring the entire waistline, for redundancy, in case any of them gives out. it is advertised as having twice the pockets as normal pants. Some are sewn shut and have no pocket in them at all, and as you reach into one of the open ones, you find it has two openings in the bottom. The pocket is a hand size pair of pants sewn into the pocket, which itself has four pockets you can fit just the edge of your pinkie into. Each of the features in isolation is plausible, and it certainly comes with a lot more of the elements of clothing, but as a garment it doesn't make a lot of sense - like it was designed by something that had never worn clothes before.
You run your hand along the material, some areas are pliant, stretchy, as expected, but invisibly it transitions to a chalky texture that crumbles and gives way. There doesn't seem to be any pattern to this, and as you stand up you find the chalky residue has stained your couch. When you crease it, it becomes dangerously sharp, and so you need to make sure to avoid bunching or handling it too roughly. The first time you unzip the zipper, it separates neatly in half into what at first seems like two pieces, but on closer inspection is a single Möbius strip, you can't fathom how it would have ever been zipped in the first place. Tightening your belt, you hear a distant tear in a part of the fabric you didn't know had tension on it. Attempting to stitch it is futile, each time a needle pierces it, the fabric seems to deflate (unclear that it was holding air at all) and decay around the puncture.
reaching for a word, AI output is.... Punctate? Human art, craft, code, writing is a surface where neighboring elements belong with each other, support each other. They have a history of creation that aligns with use, and desire paths get cut along the way - you hang a hook to hold your keys by the door because experience says you need them when you walk out the door. You finish the thoughts you started several paragraphs ago, building them into a larger idea. The AI output can produce isolated things at the scale of a context window that appear well crafted, but there is no joinery, the edges are sharp and invisible. A thousand counterfeit odd-sized legos jammed haphazardly into a sack. They degrade poorly, suddenly, and in ways that are difficult to imagine any human work would. The LLM book is a hundred two page, five paragraph essays whose table of contents insists tells a story, but each chapter appears to have been written without having read the others, full of odd repetition, conflicting, superfluous argumentation, ending with a confident flourish as if each is the end of the book.
So if the argument that median AI output is "better" than median human output isn't being made cynically, then what it betrays a profound shallowness about the ability to evaluate "quality." The code is well formatted, commented, comes with things that appear to be tests, but doesn't compose into a coherent work. The prose is grammatically correct but makes a readers skin crawl and imparts no intended meaning. The vibe coder just makes more code to cover over the shallowness, widening but never deepening similarly to how the fast fashion shopper buys a new pair of pants every week because laundering the old ones destroys them.
The deeper irony is that "total continuity" is the thing that LLMs provide as an interface and as a UX - every button already exists, and no error state is terminal. The utterly forgiving nature of being able to talk your way out of any spot, the constant reassurance that every idea is right, is exactly the affordance space that people argue is the UX virtue of LLMs. Maybe the punctate sense of local "quality" that collapses on contextual inspection is made up for by the process of creation in the mind of the AI user - the appearance of unity, completeness, is not provided by the thing that is produced, but by the means of producing it, and it is never actually experienced by itself. The natural extension of "generate an email so it can be summarized by the recipients LLM" - materials, objects, works, have a disembodied, delocalized nature that is always expected to be "filled in" by the recipient. The bugs in the software aren't bugs if they're being used by something with infinite brute force tenacity and without need for intuition. The story doesn't need to be coherent when it will be completed by each reader's personalized reality model.
That's the only way I can make sense of people claiming that the LLM output is "high quality", assuming its a truly experienced and believed feeling, which you must in order to understand how someone might believe something that seems incomprehensible to you.